Picture coding apparatus for a still picture sequence and picture decoding apparatus for a still picture sequence

ABSTRACT

A picture coding apparatus reduces a load in decoding. The picture coding apparatus codes each picture according to a picture type of the picture. The picture types include at least an I picture, a P picture, a B picture, and a skipped picture. A first coder is configured to code first supplementary information, including coded pictures and indicating respective picture types of the coded pictures, in a decoding order of the coded pictures. A second coder is configured to code second supplementary information, indicating respective pieces of picture structure information of the coded pictures, in the decoding order. A writer is configured to write, at a position prior to a starting picture, the first supplementary information coded by said first coder and the second supplementary information coded by said second coder.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Stage Application of InternationalApplication No. PCT/JP2005/018735, filed Oct. 5, 2005, and claims thebenefit of U.S. Provisional Application No. 60/616,203, filed Oct. 7,2004.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to a picture coding apparatus which codesa moving picture, a stream which is generated by an image coding methodusing the picture coding apparatus, and a picture decoding apparatuswhich decodes the stream.

(2) Description of the Related Art

Recently, with the arrival of the age of multimedia which integrallyhandles audio, video and pixel values, existing information media, forexample, newspaper, journal, Television, radio and telephone, and othermeans through which information is conveyed to people, has come underthe scope of multimedia. In general, multimedia refers to arepresentation in which not only characters but also graphic symbols,audio and especially pictures and the like are related to each other.However, in order to include the aforementioned existing informationmedia in the scope of multimedia, it appears as a prerequisite torepresent such information in digital form.

However, when estimating the amount of information contained in each ofthe aforementioned information media in digital form, the informationamount per character requires 1 to 2 bytes whereas audio requires morethan 64 Kbits per second (telephone quality), and a moving picturerequires more than 100 Mbits per second (present television receptionquality). Therefore, it is not realistic to handle the vast amount ofinformation directly in digital form via the information media mentionedabove. For example, a videophone has already been put into practical usevia Integrated Services Digital Network (ISDN) with a transmission rateof 64 Kbits/sec to 1.5 Mbits/sec, however, it is impossible to transmita picture captured by a TV camera.

This therefore requires information compression techniques, and forinstance, in the case of a videophone, video compression techniquescompliant with H.261 and H.263 Standards recommended by InternationalTelecommunication Union-Telecommunication Standardization Sector (ITU-T)are employed. According to the information compression techniquescompliant with the MPEG-1 standard, picture information as well as audioinformation can be stored in an ordinary music CD (Compact Disc).

Here, Moving Picture Experts Group (MPEG) is an international standardfor a compression of moving picture signals and the MPEG-1 is a standardthat compresses video signals down to 1.5 Mbit/s, namely, to compressthe information included in TV signals approximately down to ahundredth. The quality targeted by the MPEG-1 standard was mediumquality so as to realize a transmission rate primarily of about 1.5Mbits/sec, therefore, MPEG-2, standardized with the view to meeting therequirements of even higher quality picture, realizes a TV broadcastquality for transmitting moving picture signals at a transmission rateof 2 to 15 Mbits/sec.

In the present circumstances, a working group (ISO/IEC JTC1/SC29/WG11)previously in charge of the standardization of the MPEG-1 and the MPEG-2has further standardized MPEG-4 which achieves a compression ratesuperior to the one achieved by the MPEG-1 and the MPEG-2, allowscoding/decoding operations on a per-object basis and realizes a newfunction required by the age of multi media. At first, in the process ofthe standardization of the MPEG-4, the aim was to standardize a low bitrate coding, however, the aim is presently extended to a more versatilecoding including a high bit rate coding for interlaced pictures andothers. Moreover, the ISO/IEC and the ITU-T have jointly developed, as anext-generation image coding method, a standardization of MPEG-4Advanced Video Coding (AVC) with a higher compression rate, andcurrently Society of Motion Picture and Television Engineers (SMPTE)attempts to standardize a VC-1 (Proposed SMPTE Standard for Television:VC-1 Compressed Video Bitstream Format and Decoding Process, FinalCommittee Draft 1 Revision 6, 2005 Jul. 13). A target of the VC-1 is toextend a coding tool and the like, based on the methods of the MPEG-2and MPEG-4 standards. The VC-1 is expected to be used fornext-generation optical disk peripheral devices, such as a Blu-ray disc(BD) and a High Definition (HD) DVD.

In general, in coding of a moving picture, compression of informationvolume is performed by eliminating redundancy both in spatial andtemporal directions. Therefore, an inter-picture prediction coding,which aims at reducing the temporal redundancy, estimates a motion andgenerates a predicted picture on a block-by-block basis with referenceto prior and subsequent pictures, and then codes a differential valuebetween the obtained predicted picture and a current picture to becoded. Here, “picture” is a term to represent a single screen and itrepresents a frame when used for a progressive picture whereas itrepresents a frame or fields when used for an interlaced picture. Theinterlaced picture here is a picture in which a single frame consists oftwo fields respectively having different time. For coding and decodingan interlaced picture, three ways are possible: processing a singleframe either as a frame, as two fields or as a frame/field structuredepending on a block in the frame.

A picture to which an intra-picture prediction coding is performedwithout reference pictures is referred to as an “I-picture”. A pictureto which the inter-picture prediction coding is performed with referenceto a single picture is referred to as a “P-picture”. A picture to whichthe inter-picture prediction coding is performed by referringsimultaneously to two pictures is referred to as a “B-picture”. TheB-picture can refer to two pictures, arbitrarily selected from thepictures whose display time is either forward or backward to that of acurrent picture to be coded, as an arbitrary combination. However, thereference pictures need to be already coded or decoded as a condition tocode or decode these I-picture, P-picture, and B-picture.

FIGS. 1A and 1B are diagrams showing a structure of the conventionalMPEG-2 stream. As shown in FIG. 1B, the stream according to the MPEG-2standard has a layered system. The stream is made up of a plurality ofGroup of Pictures (GOP). It is possible to edit a moving picture and toperform random access on it by using the GOP as a basic unit used incoding processing. This means that a starting picture in the GOP is arandom access point. The GOP consists of a plurality of pictures, eachbeing I-picture, P-picture and B-picture. The stream, GOP and picturerespectively include a synchronous signal (sync) indicating a boundarybetween respective units and a header that is data commonly included inthe respective units.

FIGS. 2A and 2B are examples of a prediction structure of picturesaccording to the MPEG-2 standard. Shaded pictures in FIG. 2A arereference pictures which are referred to predict other pictures. Asshown in FIG. 2A, in the MPEG-2 standard, P-picture (picture P0, P6, P9,P12, or P15) can be predicted from one picture, either I-picture orP-picture, whose display time immediately precedes that of theP-picture. B-picture (picture B1, B2, B4, B5, B7, B8, B10, B11, B13,B14, B16, B17, B19, or B20) can be predicted from one picture whosedisplay time immediately precedes the B-picture or one picture whosedisplay time immediately follows the B-picture, both of which can beeither I-picture or P-picture. The positions of the B-pictures arearranged in the stream, either immediately subsequent to I-picture orP-picture. Therefore, at the time of performing random access, all thepictures subsequent to I-picture can be decoded and displayed, whendecoding starts from I-picture. Regarding a structure of the GOP, thepictures from I3 to B14 can be considered as one GOP, as shown in FIG.2B for example.

FIG. 3 is a diagram showing a structure of a stream according to theVC-1. The stream according to the VC-1 also has the same structure asdescribed for the MPEG-2 standard. However, a random access point isreferred to as an “entry point” which is added with an entry pointheader (Entry Point HDR). Data from the entry point to a next entrypoint is a random access unit (RAU), which is equivalent to one GOPaccording to the MPEG-2 standard. Hereafter, the RAU according to theVC-1 is referred to as a “random access point (RAU)”. Note that the RAUcan store user data regarding pictures in the RAU (user data atEntry-point level), and the RAU is arranged immediately subsequent tothe entry point header.

Here, types of pictures according to the VC-1 are described. In theVC-1, the I-picture, P-picture, and B-picture are also defined. TheseI-picture, P-picture, and B-picture have the same prediction structureas described for the MPEG-2 standard. In the VC-1, in addition to theabove three types of picture, there are two more defined types, whichare Skipped picture and BI-picture. The Skipped picture is a picturewhich does not include any pixel data, and treated as a P-picture havingthe same pixel data of a prior reference picture in decoding order. Forexample, in examples of (1) and (2), a picture S5 is regarded the samepicture as a picture P3, so that the same operation of decoding thestream is performed in both (1) and (2).

(1) Display order: Picture I0, Picture B2, Picture P1, Picture B4,Picture P3, Picture B6, Picture S5 (Note that the picture represented bya symbol including I is an I-picture, the picture represented by asymbol including P is a P-picture, the picture represented by a symbolincluding B is a B-picture, and the picture represented by a symbolincluding S is a Skipped picture. For example, the picture S6 is aSkipped picture. The numerals attached to the symbols of the picturesrepresent decoding order.)

(2) Display order: Picture I0, Picture B2, Picture P1, Picture B4,Picture P3, Picture B6, Picture P5 (P5 has the same pixel data as P3.)

The Skipped picture is especially useful when pictures are still. Forexample, in a case where the pictures are still in the middle of theRAU, Skipped pictures are used where the pictures are still, forexample, where there are picture I0, picture P1, picture P2, picture P3,picture S4, picture S5, picture S6 . . . , in order to reduce an amountof data to be coded.

Furthermore, BI-picture is a picture having characteristics of theB-picture and I-picture. More specifically, the BI-picture has theB-picture characteristics in which decoding order is different fromdisplay order, and the picture is not a reference picture for otherpictures. In addition, the BI-picture has the I-picture characteristicsin which all macroblocks are applied with an intra-picture coding andthe picture is not predicted from any other pictures.

Next, a method for distinguishing the I-picture, P-picture, B-picture,Skipped picture, and BI-pictures is described. Basically, the types ofpictures can be distinguished based on the picture types included in apicture layer in a stream. However, the picture types indicated by thepicture layer are defined as following, depending on profiles.

For example, in a simple profile, picture types are indicated asI-picture and P-picture. In a main profile, picture types are indicatedas I-picture, P-picture, and B- or BI-picture. In an advanced profile,picture types are indicated as I-picture, P-picture, B-picture,BI-picture, and Skipped picture.

Here, in both of the simple profile and the main profile, it isimpossible to distinguish the Skipped picture by using the picture typesin the picture layer, so that, in a case where an arbitrary picture hasa size of one or less byte, the picture is defined as the Skippedpicture. Furthermore, in the main profile, one picture type is definedto represent B-picture or BI-picture, so that it is impossible todistinguish B-picture from BI-picture, based on the picture type.

FIG. 4 is a block diagram showing a picture coding apparatus forrealizing the conventional image coding method.

A picture coding apparatus 800 performs compressed coding, variablelength coding, and the like, for an inputted picture signal Vin, therebytransforming the picture signal Vin into a bitstream (stream) Str to beoutputted. The picture coding apparatus 800 is comprised of a motionestimation unit 801, a motion compensation unit 802, a subtractor 803,an orthogonal transformation unit 804, a quantization unit 805, aninverse quantization unit 806, an inverse orthogonal transformation unit807, an adder 808, a picture memory 809, a switch 810, a variable lengthcoding unit 811, and a prediction structure determination unit 812.

The picture signal Vin is inputted into the subtractor 803 and themotion estimation unit 801. The subtractor 803 calculates a differentialbetween the inputted picture signal Vin and a predicted picture, andoutputs the differential to the orthogonal transformation unit 804. Theorthogonal transformation unit 804 transforms the differential into afrequency coefficient, and outputs the frequency coefficient into thequantization unit 805. The quantization unit 805 quantizes the inputtedfrequency coefficient, and outputs the resulting quantization value Qcinto the variable length coding unit 811.

The inverse quantization unit 806 inversely quantizes the quantizationvalue Qc in order to restore the original frequency coefficient, andoutputs the resulting frequency coefficient to the inverse orthogonaltransformation unit 807. The inverse orthogonal transformation unit 807performs inverse-frequency transformation on the frequency coefficientto be transformed into a pixel differential, and outputs the pixeldifferential to the adder 808. The adder 808 adds the pixel differentialwith a predicted picture which is outputted from the motion compensationunit 802, and generates a decoded picture. The switch 810 is On when thedecoded picture is instructed to be stored, and the decoded picture isstored into the picture memory 809.

On the other hand, the motion estimation unit 801, in which the picturesignal Vin is inputted in units of macroblocks, searches the decodedpictures (reference pictures) which are stored in the picture memory809, detects an image having the most similar image to a macroblockindicated by the picture signal Vin, and determines a motion vector MVfor indicating a location of the image.

The motion compensation unit 802, by using the determined motion vectorand the like, retrieves the most suitable image for a predicted picture,from the decoded picture stored in the picture memory 809.

A prediction structure determination unit 812 determines, based on a RAUstart picture Uin, that a picture to be coded is at a RAU startlocation, then instructs, using a picture type Pt, the motion estimationunit 801 and the motion compensation unit 802 to code (inter-picturecoding) the picture as a special randomly-accessible picture, andfurther instructs the variable length coding unit 811 to code thepicture type Pt.

The variable length coding unit 811 performs variable length coding onthe quantization value Qc, the picture type Pt, and the motion vector MVin order to generate a stream Str.

FIG. 5 is a block diagram showing a picture decoding apparatus 900 forrealizing the conventional image decoding method. The reference numeralsin FIG. 4 are assigned to identical units in FIG. 5, and the those unitsoperate in the same manner as described for the picture coding apparatusfor realizing the conventional image coding method in FIG. 4, so thatthe details of those units are not described herein below.

The variable length decoding unit 901 decodes the stream Str, andoutputs the quantization value Qc, a reference picture specificationinformation Ind, the picture type Pt, the motion vector MV, and thelike. The picture memory 809 obtains the movement vector MV, the motioncompensation unit 802 obtains the picture type Pt, the movement vectorMV, and the reference picture specification information Ind, and theinverse quantization unit 806 obtains the quantization value Qc. Thedecoding is performed by the picture memory 809, the motion compensationunit 802, and the inverse quantization unit 806, the inverse orthogonaltransformation unit 807, and the adder 808. The operation of thedecoding has been described with reference to the block diagram of FIG.4 showing the picture coding apparatus 800 for realizing theconventional coding method.

A buffer memory 902 is a memory for storing a decoded picture Vout whichis outputted from the adder 808, and a display unit 903 obtains thedecoded picture Vout from the buffer memory 902 and displays a pictureaccording to the decoded picture Vout. Note that the buffer memory 809and the picture memory 902 can share the same memory.

FIG. 6 is a flowchart showing decoding during special play-back, such ashigh-speed play-back, performed by the conventional picture decodingapparatus 900. Firstly, the conventional picture decoding apparatus 900detects, from the stream Str, a header of a picture to be decoded atStep S1001. Then at Step 1002, the conventional picture decodingapparatus 900 examines, based on a picture type in the header includedin the picture layer, whether or not the starting picture needs to bedecoded. At Step S1003, the conventional picture decoding apparatus 900determines whether or not the picture is examined to be decoded at Step1002, and if the decoding needs to be decoded, then the processingproceeds to Step S1004, while if the picture does not need to bedecoded, then the processing proceeds to Step S1005. Finally, at StepS1005, the conventional picture decoding apparatus 900 determineswhether or not the processing completes even for a last picture to beplay-backed, such as a last picture in a RAU or a stream, and if thereare still pictures to be processed, the processing repeats the stepsfrom Step S1001 to S1005, and if the last picture is processed, theprocessing completes.

However, in the above conventional picture coding apparatus 800 andpicture decoding apparatus 900, there is a problem of a large amount ofprocessing load, during coding the stream Str which includes Skippedpictures, and especially during the special play-back such as high-speedplay-back.

FIG. 7 is an explanatory diagram showing the problem in the aboveconventional picture coding apparatus 800 and picture decoding apparatus900.

In (a) of FIG. 7, a structure of the conventional RAU including theSkipped pictures is shown. The RAU is comprised of twenty-four picturesin which the images are still in the fourth and following pictures indecoding order, so that the fifth and later pictures are all Skippedpictures. When such a RAU is play-backed at triple speed, theconventional picture decoding apparatus 900 attempts to decode the 1st,4th, 7th, 10th, 13th, 16th, 19th and 22nd pictures, sequentially to beplay-backed. However, pictures to be practically decoded are only firstI-picture and the fourth P-picture as shown in (c) of FIG. 7.

This means that, in a RAU in the conventional stream Str, the picturedecoding apparatus 900 cannot determine whether or not the pictures areto be decoded, unless a head of each picture (picture layer) is searchedto obtain a picture type, since each picture layer includes a picturetype of the picture. Therefore, as shown in (b) of FIG. 7, the picturedecoding apparatus 900 needs to analyze the 7th, 10th, 13th, 16th, 19thand 22nd Skipped pictures to obtain the picture types.

As described above, for the high-speed play-back of the conventionalRAU, the conventional picture coding apparatus and picture decodingapparatus need to analyze even pictures which do not need to be decoded,which eventually results in a large amount of data for decoding.

Thus, the present invention addresses the above problems and an objectof the present invention is to provide a picture coding apparatus and apicture decoding apparatus which can reduce load in decoding.

SUMMARY OF THE INVENTION

In order to solve the above problems, the present invention provides apicture coding apparatus which codes a picture, the picture codingapparatus including: an coding unit operable to code each pictureaccording to a picture type of the picture; a map generation unitoperable to generate a map which indicates a still picture sequence in arandom access unit that includes a plurality of coded pictures; and awriting unit operable to write, into the random access unit, the mapwhich is generated by the map generation unit.

Thereby the map is stored in the random access unit, so that the picturedecoding apparatus can easily specify, from the map, the still picturesequence in the in random access unit. As a result, the picture decodingapparatus does not need to determine whether or not the picture is aSkipped picture, by analyzing a plurality of the picture layers one byone which are included in the random access units as in the conventionalmethod, so that it is possible to reduce the load in decoding.

Further, the writing unit may be operable to write the map at a positionwhich is prior to a starting picture in the random access unit.

Thereby the picture decoding apparatus obtains the random access unitfrom the starting of the random access unit, thereby enabling to easilyand speedily detect the map, so that it is possible to reduce the loadin decoding.

Still further, the map generation unit may be operable to generate themap which indicates the picture type of each picture which is includedin the random access unit. For example, the picture type may indicatewhether or not a current picture is a Skipped picture which is to bedisplayed with an image of a reference picture that is positionedimmediately prior to the Skipped picture in decoding order.

Thereby the picture decoding apparatus can specify as the still picturesequence, by using a picture type of each picture which is indicated inthe map, a range in which a plurality of Skipped pictures follow afteran I-picture or P-picture.

Still further, the map generation unit may be operable to generate themap which indicates a starting picture and a last picture in the stillpicture sequence.

Thereby the picture decoding apparatus can easily specify the stillpicture sequence, according to the starting and last pictures which areindicated in the map.

Here, in order to achieve the above object, the present inventionprovides a picture decoding apparatus which decodes a random access unitthat includes a plurality of coded pictures, the picture decodingapparatus includes: a detection unit operable to detect, from the randomaccess unit, a map which indicates a still picture sequence in therandom access unit; a selection unit operable to select a picture to bedecoded, from the coded pictures in the random access unit, based on thestill picture sequence which is indicated by the map detected by thedetection unit; and a decoding unit operable to decode the picture whichis selected by the selection unit.

Thereby the picture to be decoded is previously selected based on thestill picture sequence, prior to decode the picture, so that it is notnecessary, as in the conventional method, to determine whether or notthe picture is a Skipped picture, by analyzing a plurality of picturelayers one by one which are included in the random access unit whiledecoding, which can reduce the load in decoding.

Moreover, in order to achieve the object, the present invention providesa coded picture signal which includes a plurality of coded pictures foreach random access unit, the image coding signal comprising a mapindicating a still picture sequence in random access unit for eachrandom access unit.

Thereby the map is stored in the random access unit, so that the picturedecoding apparatus can easily specify, from the map, the still picturesequence in the pictures in the random access unit. As a result, thepicture decoding apparatus does not need to determine whether or not thepicture is a Skipped picture, by analyzing the plurality of the picturelayers one by one which are included in the random access unit, as inthe conventional method, so that it is possible to reduce the load indecoding.

Furthermore, the map may be stored at a position prior to any pictureswhich are included in the random access unit.

Thereby the picture decoding apparatus obtains the random access unitsfrom the beginning of the random access unit, thereby enabling to easilyand speedily detect the random access units, so that it is possible toreduce the load in decoding.

Note that the present invention can be realized not only as the abovedescribed picture coding apparatus, picture decoding apparatus, andimage coding signal, but also as an image coding method, an imagedecoding method, a program, a storage medium which stores the program,and an integrated circuit which includes the above devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are diagrams showing a structure of the MPEG 2 stream.

FIGS. 2A and 2B are diagrams showing examples of a prediction structurebetween pictures used in the MPEG 2 standard.

FIG. 3 is a diagram showing a structure of the conventional VC-1 stream.

FIG. 4 is a block diagram showing a structure of the conventionalpicture coding apparatus.

FIG. 5 is a block diagram showing a structure of the conventionalpicture decoding apparatus.

FIG. 6 is a flowchart showing operations which are performed by theconventional picture coding apparatus.

FIG. 7 is a diagram showing a problem in a stream which is generated bythe conventional picture coding apparatus, during high-speed play-back.

FIG. 8 is a diagram showing an example of a structure of a RAU which isincluded in a VC-1 stream according to the first embodiment of thepresent invention.

FIG. 9A is a diagram showing an example of a syntax of a RAU map MI.

FIG. 9B is a diagram showing another example of the syntax of the RAUmap MI.

FIG. 9C is a diagram showing still another example of the syntax of theRAU map MI.

FIG. 9D is a diagram showing still further example of the syntax of theRAU map MI.

FIG. 10 is a block diagram showing a structure of the picture decodingapparatus according to the first embodiment of the present invention.

FIG. 11 is a flowchart showing operations which are performed by thepicture decoding apparatus according to the first embodiment of thepresent invention.

FIG. 12 is a flowchart showing operations of analyzing a RAU map whichare performed by the picture decoding apparatus according to the firstembodiment of the present invention.

FIG. 13A is a diagram showing a RAU which is play-backed at a high speedby the picture decoding apparatus according to the first embodiment ofthe present invention.

FIG. 13B is a diagram showing a RAU map MI according to FIG. 13A.

FIG. 13C is a flowchart showing operations of play-backing at a highspeed a stream STR having the RAU in FIG. 13A, which is performed by thepicture decoding apparatus according to the first embodiment of thepresent invention.

FIG. 14 is an explanatory diagram showing a play-back method which isperformed by a picture decoding apparatus according to a variation ofthe first embodiment.

FIG. 15 is a flowchart showing the play-back method which is performedby the picture decoding apparatus according to the variation of thefirst embodiment.

FIG. 16 is a block diagram showing a structure of a picture codingapparatus according to the second embodiment of the present invention.

FIG. 17 is a flowchart showing operations which is performed by thepicture coding apparatus according to the second embodiment of thepresent invention.

FIGS. 18A and 18B are diagrams showing a prediction structure of aB-skip picture.

FIGS. 19A, 19B, and 19C are explanatory diagrams showing a storagemedium which stores a program for realizing an image coding method andan image decoding method according to the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

The following describes embodiments according to the present inventionwith reference to the drawings.

First Embodiment

A RAU map is stored at the beginning of a RAU in a VC-1 stream accordingto the first embodiment of the present invention, and a picture decodingapparatus according to the first embodiment specifies a still picturesequence in the RAU by analyzing the RAU map.

FIG. 8 is a diagram showing an example of a structure of the RAU whichis included in the VC-1 stream according to the first embodiment.

The RAU structure includes an entry point header (Entry Point HDR) anduser data which are positioned at the beginning of the RAU, and aplurality of pictures which follow the user data. Note that, in the VC-1standard, the RAU is referred to as an entry point segment (EPS).

More specifically, the RAU according to the first embodiment differsfrom the conventional RAU in that the RAU includes a RAU map MI which isarranged in the user data (user data at Entry-point level) and indicatesif Skipped pictures are present in the RAU, also specifies a stillpicture sequence in the RAU.

Therefore, the picture decoding apparatus according to the firstembodiment can examine, by referring to the RAU map MI, whether or notthe RAU includes any Skipped pictures and can specify the still picturesequence, so that it is possible to specify, without analyzing eachpicture layer in the RAU, pictures which do not need to be decoded,which results in reducing an amount of data to be decoded.

FIG. 9A is a diagram showing an example of a syntax of the RAU map MI.

num_pic_in_RAU represents the number of pictures in the RAU.frame_field_flag represents whether each picture in the RAU is coded ina field structure or in a frame structure. pic_type represents a picturetype (including a Skipped picture type) of each picture. Note that theinformation regarding each picture is indicated in decoding order. Thismeans that the RAU map MI specifies a still picture sequence in the RAU,by indicating the picture types (including a Skipped picture type) ofthe pictures in each RAU. Here, the still picture sequence in the firstembodiment means a position and a range from a reference picture to alast Skipped picture, in a case where a sequence of a plurality ofSkipped pictures follows the reference picture (I-picture or P-picture)in decoding order.

For example, the RAU map MI indicates that pictures from the secondpicture to a last picture in the RAU are all Skipped pictures. In theabove case, by referring to pic_type in the RAU map MI, the picturedecoding apparatus determines to decode the starting picture and displaythe result repeatedly, without decoding the second and followingpictures.

Note that the RAU map MI can include further information on 3:2 pulldownwhich indicates how many fields one frame is equivalent to in beingdisplayed, or whether decoding of the frame starts from a top field or abottom field, and the like, for each picture.

FIG. 9B is a diagram showing another example of the syntax of the RAUmap MI.

In the advanced profile in the VC-1 standard, picture types of the firstfield and the second field for a frame of field-structure are indicatedby a field-picture type which is included in the picture layer. Thefield-picture type (picture types of the first picture and the secondpicture) is defined by eight patterns which are (I, I), (I, P), (P, I),(P, P), (B, B), (B, BI), (BI, B), and (BI, BI). Therefore, in a casewhere a picture consists of fields, it is possible to indicate picturetypes of both of the first field and the second field which are includedin a frame, by indicating the field-picture type.

Therefore, the syntax of the RAU map MI shown in FIG. 9B also indicatesa field-picture type of a picture, in a case where the picture consistsof fields. More specifically, num_frame_in_RAU represents the number ofthe frames in the RAU. field_coding_flag represents whether or not thepicture consists of fields. In a case where the picture consists offields, a field-picture type of the picture is represented byfield_type_flag, and in a case where the picture does not consists offields, a picture type of the picture is represented by picture_type.

That is, the RAU map MI, in the same manner as the RAU map MI shown inFIG. 9A, specifies a still picture sequence in a RAU, by indicatingpicture types of every pictures in each RAU.

Moreover, in a case where the RAU includes only I-pictures and Skippedpictures, or only I-pictures, P-pictures, and Skipped pictures, a partor all parts of the RAU becomes a still picture sequence. In this case,depending on whether a processed part is the still picture sequence or anormal moving-picture sequence, the picture coding apparatus changes thedecoding and displaying processing, so that the RAU map MI may includefurther information regarding whether or not the RAU includes any stillpicture sequence.

FIG. 9C is a diagram showing still another example of the syntax of theRAU map MI.

In this syntax, motionless_flag represents whether or not the RAUincludes any still picture sequence, and start_pic_num and end_pic_numspecify the still picture sequence in the RAU. More specifically, in acase where motionless_flag is 1, the RAU map MI indicates that the RAUincludes a sill picture sequence. Further, in a case wheremotionless_flag is 1, the RAU map MI indicates that the still picturesequence starts with an I-picture or P-picture which is represented bystart_pic_num, and ends with a Skipped picture which is represented byend_pic_num.

Note that it is possible to set motionless_flag to as 1, only in a casewhere all parts of the RAU are a still picture sequence or where the RAUincludes a still picture sequence which continues longer than a certaintime period.

FIG. 9D is a diagram showing other example of the syntax of the RAU mapMI.

In this syntax, number_of_pictures_in_EPS represents the number ofpictures included in the EPS. picture_structure represents whether apicture is a field or a frame, or represents how many fields one frameis equivalent to in being displayed. picture_type represents whichpicture type, namely I-picture, P-picture, B-picture, Skipped picture,or the like, the picture belongs to. Further, stuffing_bits is used toalign all bits of stuffing_bits, picture_structure, and picture_type, byintegral multiplication of eight bits. Furthermore, in this syntax,stuffing_bits, picture_structure, and picture_type are indicated indecoding order, regarding respective pictures included in the EPS.

Such RAU map MI, in the same manner as the RAU map MI shown in FIG. 9A,specifies a still picture sequence in pictures in a RAU, by indicatingpicture types of the pictures in each RAU (EPS).

Note that, the RAU map MI may store the information regarding respectivepictures in an order of displaying the pictures. Note also that the RAUmap MI may store further information which indicates whether theinformation regarding respective pictures are stored in the decodingorder or in the display order.

Note also that the RAU map MI may be stored in user data in a layer thatis different from an entry point layer, for example, in user data for astarting picture. Note also that, in a case where the RAU does notinclude any Skipped pictures, the RAU map MI does not need to begenerated. In such a case, it is possible to indicate whether or not theRAU includes any Skipped pictures, by examining the existence of the RAUmap MI.

FIG. 10 is a block diagram showing a picture decoding apparatus 100 inthe first embodiment.

The picture decoding apparatus 100 of the first embodiment which decodesthe stream STR that includes the RAU shown in FIG. 8 is comprised of:the variable length decoding unit 101, the picture memory 102, themotion compensation unit 103, the inverse quantization unit 104, theinverse orthogonal transformation unit 105, the buffer memory 106, thedisplay unit 107, the adder 108, a stream extraction unit 109, and aninformation obtainment unit 110.

This picture decoding apparatus 100 differs from the conventionalpicture decoding apparatus 900 in that the stream extraction unit 109and the information obtainment unit 110 are added.

The information obtainment unit 110 obtains the RAU map MI from thevariable length decoding unit 101, and also obtains, from the outside, aplay-back mode signal TM for instructing details of special play-backsuch as high-speed play-back. Then, the information obtainment unit 110analyzes the RAU map MI based on the play-back mode signal TM, anddetermines (selects) pictures to be decoded. The information obtainmentunit 110 outputs a decoding picture instruction signal SP whichindicates the determination results, to the stream extraction unit 109.

For example, in a case where the RAU map MI includes the syntax shown inFIG. 9C, the information obtainment unit 110 determines, based onmotionless_flag, whether or not the RAU to be play-backed includes anystill picture sequence. Then, if the determination is made that the RAUincludes a still picture sequence, the information obtainment unit 110specifies the still area, based on start_pic_num and end_pic_num. Afterspecifying the still picture sequence, the information obtainment unit110 determines, from the pictures to be play-backed which are indicatedby the play-back mode signal TM, only pictures which are not included inthe still picture sequence, as pictures to be decoded, and theinformation obtainment unit 110 outputs the determination results to thedecoding picture instruction signal SP. However, if the pictures to beplay-backed which are indicated by the play-back mode signal TM includea picture in the still picture sequence, the starting picture of thestill picture sequence is determined to be the picture to be decoded.

Further, if the RAU map MI includes the syntax shown in FIG. 9D, theinformation obtainment unit 110 specifies a still picture sequence,based on picture_type which is indicated for each picture in the RAU.Then, the information obtainment unit 110 determines, from the picturesto be play-backed which are indicated by the play-back mode signal TM,only pictures which are not included in the still picture sequence, aspictures to be decoded, and the information obtainment unit 110 outputsthe determination results to the decoding picture instruction signal SP.However, as described above, if the pictures to be play-backed which areindicated by the play-back mode signal TM include a picture in the stillpicture sequence, the starting picture of the still picture sequence isdetermined to be the picture to be decoded.

After obtaining the stream STR, the stream extraction unit 109 firstlydetects, for each RAU, the coded RAU map MI which is positioned at thebeginning of the RAU, and outputs the RAU map MI to the variable lengthdecoding unit 101. After obtaining the decoding picture instructionsignal SP which is outputted from the information obtainment unit 110based on the RAU map MI, the stream extraction unit 109 extracts, fromthe stream STR, data of the pictures to be decoded which are indicatedby the decoding picture instruction signal SP, and outputs the data tothe variable length decoding unit 101.

When the variable length decoding unit 101 obtains the coded RAU map MIfrom the stream extraction unit 109, the variable length decoding unit101 performs variable length decoding on the coded RAU map MI, andoutputs the decoded RAU map MI to the information obtainment unit 110.Further, when the variable length decoding unit 101 obtains, from thestream extraction unit 109, the data of the pictures which are includedin the stream STR, the variable length decoding unit 101 performsvariable length decoding on the data, and outputs a quantization valueQc, a reference picture specification information Ind, a picture typePt, and a motion vector MV.

The motion compensation unit 103 retrieves an image which is indicatedby the motion vector MV, from the decoded picture (reference picture)which is stored in the picture memory 102 and indicated by the referencepicture specification information Ind, and outputs the image as apredicted picture to the adder 108.

The inverse quantization unit 104 inversely quantizes the quantizationvalue Qc to be restored as a frequency coefficient, and outputs thefrequency coefficient into the inverse orthogonal transformation unit105. The inverse orthogonal transformation unit 105 performsinverse-frequency transformation on the frequency coefficient to betransformed into a pixel differential, and outputs the pixeldifferential to the adder 108. The adder 108 adds the pixel differentialwith the predicted picture which is outputted from the motioncompensation unit 103, and generates a decoded picture Vout. Then, theadder 108 stores the decoded picture Vout into the picture memory 102and the buffer memory 106. The display unit 107 obtains the decodedpicture Vout from the buffer memory 106, and displays a picturecorresponding to the decoded picture Vout. Note that the picture memory102 and the buffer memory 106 may share a single memory.

Note also that the stream extraction unit 109 may output data of allpictures which are included in the RAU, into the variable lengthdecoding unit 101. In this case, the variable length decoding unit 101selects, from all pictures included in the RAU, pictures which need tobe decoded, based on the decoding picture instruction signal SP which isoutputted from the information obtainment unit 110. Then, the variablelength decoding unit 101 performs variable length decoding on data ofthe selected pictures. Note that the information obtainment unit 110 mayspecify the picture to be decoded only for special play-back, such ashigh-speed play-back and inverse play-back. In case of normal play-back,it can be determined to decode all the pictures without analyzing theRAU map.

FIG. 11 is a flowchart showing operations which are performed by thepicture decoding apparatus 100 according to the first embodiment.

When the picture decoding apparatus 100 receives an instruction to startspecial play-back, the picture decoding apparatus 100 firstly determineswhether or not the RAU map MI is stored in user data in an entry pointlayer (Step S100). In other words, the picture decoding apparatus 100determines whether or not the RAU map MI is detected. If the picturedecoding apparatus 100 detects the RAU map MI (YES at Step S100), thenthe processing proceeds to Step S102, and if not (NO at Step S100), thenthe processing skips directly to Step S106.

More specifically, if the picture decoding apparatus 100 detects the RAUmap MI (YES at Step S100), the picture decoding apparatus 100 analyzesthe RAU map MI (Step S102), and determines (selects), from the picturesin the RAU which are to be play-backed during special play-back,pictures to be decoded, based on result of the analysis (Step S104).

Note that, when the special play-back of the RAU starts, the picturedecoding apparatus 100 always detects the RAU map MI at Step S100, andspecifies pictures in the RAU to be decoded. In other words, when thespecial play-back of the RAU starts, the picture decoding apparatus 100in the first embodiment selects, based on the RAU map MI, from thepictures which are included in the RAU and to be play-backed during thespecial play-back, pictures except Skipped pictures, as the pictures tobe decoded.

In case the RAU map MI is not detected at Step S100, or after thepictures to be decoded are specified at Step S104, the picture decodingapparatus 100 detects a header of the picture (start code) in thepictures which are in the RAU and to be play-backed during the specialplay-back (Step S106).

Next, the picture decoding apparatus 100 examines whether or not thepicture whose header has been detected at Step 106 and which is apicture to be play-backed during the special play-back among thepictures that have been specified to be decoded at Step S104 (StepS108). Here, if the determination is made that the picture is among thepictures which have been specified to be decoded (YES at Step S108), thepicture decoding apparatus 100 decodes the picture (Step S110).

In case determination is made that the picture is not among the pictureswhich have been specified to be decoded at Step S104 (NO at Step S108),or after the picture is decoded at Step S110, the picture decodingapparatus 100 examines whether or not there are still any pictures to beprocessed (Step S112).

If no picture to be processed is found (NO at Step S112), then thepicture decoding apparatus 100 completes all operations, and if there isstill pictures to be processed (YES at Step S112), then the picturedecoding apparatus 100 repeats the operations from Step S100. Forexample, in a case where the RAU map MI has been detected at Step S100in the previous processing, and the following processing proceeds toStep 100 for the same RAU, the picture decoding apparatus 100 does notneed to detect the RAU map MI at S100 (NO at Step S100), but performsthe operation at Step S106, namely, detects a header of the next pictureto be play-backed during the special play-back.

As described above, the image decoding method in the first embodimentdiffers from the conventional image decoding method in that theoperations from Step S100 to Step S104 are included.

FIG. 12 is a flowchart showing operations of analyzing the RAU map MIwhich are performed by the picture decoding apparatus 100 according tothe first embodiment.

For example, in a case where the RAU map MI includes the syntax shown inFIG. 9D, the picture decoding apparatus 100 firstly analyzes the RAU mapMI, and specifies I-pictures, P-pictures, and Skipped pictures, from thepictures which are included in the RAU and to be play-backed during thespecial play-back (Step S120).

Next, in a case where the picture to be play-backed during the specialplay-back is a Skipped picture, the picture decoding apparatus 100determines to use a result of decoding an I-picture or a P-picture whichis immediately prior to the Skipped picture in decoding order, as apicture corresponding to the Skipped picture (Step S122).

Note that, even in normal play-back which is not the special play-back,it is possible to specify, by referring to the RAU map MI, Skippedpicture and the like, when the play-back of the RAU starts.

When the Skipped picture included in the RAU is displayed, the picturedecoding apparatus 100 displays the result of decoding the I-picture orthe P-picture which is specified at Step S122 and is immediately priorto the Skipped picture.

Here, with reference to FIGS. 13A, 13B, and 13C, operations ofhigh-speed play-back which are performed by the picture decodingapparatus according to the first embodiment.

FIG. 13A is a diagram showing a RAU which is play-backed at a highspeed.

The first picture counted from the beginning of the pictures is anI-picture, the second and third pictures are B-pictures, and the fourthpicture is a P-picture. The fifth and following pictures are all Skippedpictures. Note that all of the pictures are frames.

FIG. 13B is a diagram showing a RAU map MI which corresponds to FIG.13A. The RAU map MI includes the syntax shown in FIG. 9A. Here, allpictures are frames, so that frame_field_flag are set to as 1 for allpictures. Further, pic_type is set to as I-picture, P-picture, B-pictureor Skipped picture, for each picture. Note that, in FIG. 13B, pic_typeis set to as “I”, “P”, “B”, or “Skipped”, but, in actual practice,pic_type can also be set to as a numeric value which represents thepicture type.

FIG. 13C is a flowchart showing operations of high-speed play-back of astream STR that includes the RAU in FIG. 13A, which is performed by thepicture decoding apparatus 100 according to the first embodiment.

Firstly, the picture decoding apparatus 100 determines to play-back attriple speed the RAU in FIG. 13A which is included in the stream STR(Step S130). Note that the play-back at triple speed is a commonhigh-speed play-back, and is the same processing by which onlyI-pictures and P-pictures are play-backed, in a case where a streamstructure of the RAU includes I-picture, B-picture, B-picture,P-picture, B-picture, B-picture, P-picture, B-picture, B-picture, . . .in decoding order.

Next, the picture decoding apparatus 100 determines, based on a resultof analyzing the RAU map MI shown in FIG. 13B, that pictures from thefifth picture to the twenty-fourth picture are all Skipped pictures andthat a range from the fourth picture to the twenty-fourth picture is astill picture sequence. Then, the picture decoding apparatus 100determines to decode only first and fourth pictures, since a result ofdecoding the fourth picture is used as pictures to be displayed for thefifth and following pictures (Step S132). Subsequently, the picturedecoding apparatus 100 decodes and displays the first and fourthpictures (Step S134). Furthermore, the picture decoding apparatus 100displays the result of decoding the fourth picture repeatedly instead ofresults of decoding the seventh, tenth, thirteenth, sixteenth,nineteenth, and the twenty-second pictures.

Note that the first embodiment has described that each RAU of the VC-1stream includes a RAU map and that the picture decoding apparatus 100decodes the stream, but it is possible to apply any coding method tocode the stream, besides the MPEG-4AVC and the MPEG-2 standards, as faras the stream includes the RAU map. Here, even if a coding method inwhich the same picture type as Skipped picture is not defined isapplied, the method can distinguish a picture from other pictures byregarding the picture as a Skipped picture in the RAU map, as far as atype of the picture is actually the same as Skipped picture.

(Variation)

The following describes a variation of a play-back method which isperformed by the picture decoding apparatus 100 according to the firstembodiment.

For example, there would be a case that decoding of the starting picturein the still picture sequence does not complete within a decoding timeperiod which ranges from a decoding time stamp (DTS) to a presentationtime stamp (PTS). Therefore, in the variation of the first embodiment,even if the decoding of the starting picture has not completed by thePTS, the starting picture is displayed after the decoding completes.

FIG. 14 is an explanatory diagram showing the play-back method which isperformed by the picture decoding apparatus according to the variationof the first embodiment.

DTS2 represents a decoding time stamp which is included in a header of apacket (referred to as a PES packet) having a code of a starting picturepic2 in a still picture sequence, in other words, represents a time ofdecoding the starting picture pic2. PTS2 represents a presentation timestamp which is included in the header of the packet having the code ofthe starting picture pic2, in other words, represents a time ofpresentation (output or display) of the starting picture pic2. DTS1,PTS1 and PTS3 represent respective times in the same manner as describedabove.

For example, the picture decoding apparatus 100, as shown in FIG. 14,starts decoding the starting picture pic2 at DTS2. However, there is acase that a decoding completion time is after the PTS2. Therefore, in acase where a decoding completion time for the starting picture in thestill picture sequence is after PTS2, the picture decoding apparatus 100according to the variation of the first embodiment starts presentationat a time of a frame-grid which is immediately after the decodingcompletion time.

Thus, in a case where the decoding starts at a decoding time stamp whichis included in the coded starting picture, but the decoding has notcompleted by a presentation time stamp, the picture decoding apparatus100 according to the variation of the first embodiment adds a margin tothe presentation time stamp and displays the decoded starting picture atsuch presentation time stamp with the margin.

FIG. 15 is a flowchart showing the play-back method which is performedby the picture decoding apparatus 100 according to the variation of thefirst embodiment.

The picture decoding apparatus 100 according to the variation of thefirst embodiment starts decoding a starting picture at a DTS of thestarting picture in the still picture sequence (Step S140). Then, thepicture decoding apparatus 100 determines whether or not the decodinghas completed by a PTS of the starting picture (Step S142). Here, if thedetermination is made that the decoding has completed (YES at StepS142), then the picture decoding apparatus 100 displays the decodedstarting picture, at the PTS (Step S144). On the other hand, if thedetermination is made that the decoding has not yet completed (NO atStep S142), then the picture decoding apparatus 100 displays the decodedstarting picture, at a time immediately after the PTS, namely, at a timeof a frame-grid immediately after completing the decoding (Step S146).

Thus, according to the play-back method which is performed by thepicture decoding apparatus 100 of the variation of the first embodiment,in a case where the decoding of the starting picture in the stillpicture sequence is delayed and has not completed by the PTS, a displaytime of the starting picture is also able to be delayed, so that it ispossible to improve picture quality in the sill picture sequence,compared to a case where the starting picture is not displayed.

Second Embodiment

FIG. 16 is a block diagram showing a picture coding apparatus accordingto the second embodiment of the present invention.

The picture coding apparatus 200 according to the second embodiment iscomprised of: a motion estimation unit 201, a motion compensation unit202, a subtractor 203, an orthogonal transformation unit 204, aquantization unit 205, an inverse quantization unit 206, an inverseorthogonal transformation unit 207, an adder 208, a picture memory 209,a switch 210, a variable length coding unit 211, a prediction structuredetermination unit 212, and an information generation unit 213.

The motion estimation unit 201 obtains an image signal Vin in units ofmacroblocks. Then, the motion estimation unit 201 searches decodedpictures (reference pictures) which are stored in the picture memory209, and detects an image having the most similar image to a macroblockindicated by the picture signal Vin. The motion estimation unit 201determines a motion vector MV which indicates a location of the imageand outputs the vector MV. The motion estimation unit 201 outputs areference picture specification information Ind which indicates adecoded picture that has been used to detect the motion vector MV.

The motion compensation unit 202 retrieves the image which is indicatedby the motion vector MV, from the decoded pictures which are stored inthe picture memory 209 and indicated by the reference picturespecification information Ind, and outputs the image as a predictedpicture.

The picture prediction structure determination unit 212 determines,based on a RAU start picture Uin, that a picture to be coded is at a RAUstart position, then instructs, using a picture type Pt, the motionestimation unit 801 and the motion compensation unit 802 to code(inter-picture coding) the picture as a randomly-accessible picture, andfurther instructs the variable length coding unit 811 to code thepicture type Pt. More specifically, the prediction structuredetermination unit 212 specifies a picture type, for example, I-picture,P-picture, B-picture, Skipped picture, or the like, for each picture tobe coded which is included in the picture signal Vin.

The subtractor 203 obtains the picture signal Vin and the predictedpicture, then calculates a differential between the picture signal Vinand the predicted picture, and outputs the differential to theorthogonal transformation unit 204. The orthogonal transformation unit204 transforms the differential into a frequency coefficient, andoutputs the frequency coefficient into the quantization unit 205. Thequantization unit 205 quantizes the frequency coefficient which isinputted from the orthogonal transformation unit 204, and outputs theresulting quantization value Qc into the variable length coding unit211.

The inverse quantization unit 206 inversely quantizes the quantizationvalue Qc in order to restore the original frequency coefficient, andoutputs the resulting frequency coefficient to the inverse orthogonaltransformation unit 207. The inverse orthogonal transformation unit 207performs inverse-frequency transformation on the frequency coefficientto be transformed into a pixel differential, and outputs the pixeldifferential to the adder 208. The adder 808 adds the pixel differentialwith the predicted picture which is outputted from the motioncompensation unit 202, and generates a decoded picture. The switch 210is On when the decoded picture is instructed to be stored, and thedecoded picture is stored into the picture memory 209.

The information generation unit 213 generates a RAU map MI as shown inone of FIGS. 9A to 9D, according to the picture type Pt which isspecified by the prediction structure determination unit 212, andoutputs the generated RAU map MI to the variable length coding unit 211.

The variable length coding unit 211 performs variable length coding onthe quantization value Qc, the picture type Pt, the RAU map M, themotion vector MV, and the like, in order to generate a stream STR.

As described above, the picture coding apparatus 200 according to thesecond embodiment differs from the conventional picture coding apparatus800 in that the information generation unit 213 is included.

FIG. 17 is a flowchart showing operations which are performed by thepicture coding apparatus 200 according to the second embodiment.

Firstly, the picture coding apparatus 200 determines, by using theprediction structure determination unit 212, whether or not a picture tobe coded is a starting picture in a RAU (Step S200). Here, if thedetermination is made that the picture is the starting picture in theRAU (YES at Step S200), the picture coding apparatus 200 performs, byusing the variable length coding unit 211, initialization processing togenerate the RAU map MI, and obtains an area for storing the RAU map MIin a user data of an entry point layer (Step S202).

Furthermore, the picture coding apparatus 200 determines, by using theprediction structure determination unit 212, whether or not the pictureto be coded as a Skipped picture (Step S204). Here, if the determinationis made that the picture is not be a Skipped picture (NO at Step S204),then the picture coding apparatus 200 codes pixel data of the picture tobe coded (Step S206).

Then, the picture coding apparatus 200 generates and updates, by theinformation generation unit 213, a RAU map MI, based on a result of thedetermination at Step S204 (Step S208).

For example, the picture coding apparatus 200 generates the RAU map MIas shown in FIG. 9D, in order to include a picture type of the picturewhich is coded at Step S206, information indicating whether the pictureis a field or a frame, and the like. The picture coding apparatus 200may also generate the RAU map MI as shown in FIG. 9C, in order toinclude an indication of a still picture sequence.

Next, the picture coding apparatus 200 determines whether or not thepicture determined at Step S204 is a last picture in the RAU (StepS210). In other words, the picture coding apparatus 200 determineswhether or not the processing has been performed for all pictures whichare included in the RAU. Here, if the determination is made that thepicture is a last picture (YES at S210), then the picture codingapparatus 200 specifies and codes the RAU map MI, by using the variablelength coding unit 211, and writes the RAU map MI into the area which isobtained at Step S202 (Step S212).

Then, the picture coding apparatus 200 determines whether or not thereare still pictures to be processed, among pictures included in thestream STR (Step S214). Here, if the determination is made that there isstill a picture to be processed (YES at Step S214), then the picturecoding apparatus 200 repeats operations from Step S200, and if thedetermination is made that no picture to be processed is found (NO atStep S214), then the picture coding apparatus 200 completes all codingoperations.

Note that, in a case where the information regarding the RAU map MI isnot known, or a case where a buffer memory is added in order to bufferdata of pictures which are included in the RAU, it is possible to skipStep S202. In this case, the storage area for the RAU map MI is obtainedat Step S212, and the RAU map MI is stored in the user data of the entrypoint layer.

Note also that the picture coding apparatus 200 may generate the streamSTR which includes Skipped pictures, with a fixed bit rate. The amountfor coding one Skipped picture is about 1 byte, and it is necessary toadjust a size of the stream STR by inserting padding data, when codingthe picture signal Vin with a fixed bit rate. Here, the padding data maybe inserted only in Skipped pictures. Thereby, it is possible to decodethe picture without consuming time for processing the padding data whichis inserted in a slice of the data in the picture.

Note also that a sequence layer and the information of the entry pointlayer need to be read out firstly during the special play-back, so it isdesirable to downsize the data as much as possible. Therefore, it can bedetermined not to insert the padding data between the sequence layer andthe entry point layer.

It is also possible to multiplex and record the stream STR which isgenerated by the coding method according to the second embodiment,together with audio data. Examples of the multiplexing method are amethod which is standardized for each packaged media and the like, suchas a method using a transport stream packet of the MPEG-2 system or apacket which is defined in Blu-ray Disc (BD).

Moreover, in the simple profile and the main profile, Skipped picturecannot be identified by the picture type in the picture layer.

Therefore, even if the picture type in the picture layer for eachpicture is I-picture, P-picture, B-picture, or BI-picture, the picturecoding apparatus 200 according to the second embodiment may examine,based on a size of the picture, whether or not the picture is a Skippedpicture, and if the picture is a Skipped picture, then the RAU map MImay be generated to indicate that the picture is a Skipped picture. Thismeans that the RAU map MI in the stream STR which is generated by thepicture coding apparatus 200 indicates picture types of respectivepictures including Skipped pictures, even in the simple profile and themain profile.

From the same reason, in the main profile, it is also impossible todistinguish a B-picture and a BI-picture by the picture type in thepicture layer.

Therefore, the picture coding apparatus 200 according to the secondembodiment generates the RAU map MI which indicates picture types ofrespective pictures which are included in the RAU, based on the picturetypes Pt which are specified by distinguishing a B-picture and aBI-picture by the prediction structure determination unit 212. Thismeans that the RAU map MI in the stream STR which is generated by thepicture coding apparatus 200 can distinguish B-pictures and BI-pictures,even in the main profile. The BI-picture, not like B-picture, can beindependently decoded, so that the distinguishing of Bi-picture andB-picture increases flexibility to select pictures to be decoded andplay-backed during the special play-back.

In a case a sequence layer is present, it is possible that the RAUinclude the sequence layer, for example, by always adding the sequencelayer to the entry point layer. Further, the user data in the entrypoint layer may include further information besides the RAU map MI.

In the special play-back, it is important to specify a picture to bedecoded and to efficiently access such picture. Therefore, the RAU mapMI may indicate address information regarding each picture. Here, theaddress information may be information regarding a byte position countedfrom the beginning of the RAU or information to specify a packet inwhich each picture is stored when the coded data is packetized by atransport stream packet, and the like. Note that the address informationmay be added, not for all pictures, but only for pictures to be decodedduring the special play-back, such as I-pictures or P-pictures.

<Variation>

The following describes a variation of Skipped picture according to thesecond embodiment.

In the second embodiment, a Skipped picture is generated to be aP-picture which has the same pixel data of the reference pictureimmediately prior to the Skipped picture in decoding order. Thereby, theSkipped picture cannot be used instead of a B picture.

Thus, the variation of the second embodiment generates the Skippedpicture as a B-picture which is not predicted from any other pictures,and as a picture having the same pixel data as a picture that is thereference picture immediately prior to the Skipped picture in displayorder (hereinafter, referred to as a B-skip picture). More specifically,in the variation of the second embodiment, by newly introducing theB-skip picture, it is possible to form a GOP structure, such asI-picture, B-picture, B-picture, P-picture, B-picture, B-picture,P-picture, B-picture, B-picture, . . . , which is commonly used in theMPEG-2 standard, so that IP play-back (special play-back forplay-backing only I-pictures and P-pictures) can be easily realized inthe picture decoding apparatus.

FIGS. 18A and 18B show examples in which the B-skip pictures are used.Note that, in FIGS. 18A and 18B, “I”, “B”, “P” and “B-skip” which areincluded in codes “I2”, “B0”, “P5” and “B-skip6” and the like, representpicture types of respective pictures, and numerals which are added tothe picture types indicate a display order. Note that, in FIG. 18A, thepictures in the RAU are arranged in decoding order, and in FIG. 18B, thepictures in the RAU are arranged in display order. A picture B-skip6 anda picture B-skip7 are predicted from only a picture P5 which is areference picture immediately prior to the pictures in display order,and not predicted from a picture P8. The picture coding apparatus 200according to the variation of the second embodiment generates a streamSTR having the RAU shown in FIGS. 18A and 18B.

Here, whether or not a picture is a B-skip picture is indicated by apicture type in the RAU map MI. On the other hand, even if the pictureis a B-skip picture, a picture type or a field-picture type which isincluded in a picture layer of the picture is a B-picture. Therefore,such a stream STR can maintain compatibility with the conventionalstream, so that even the conventional decoder which cannot analyze theRAU map MI can treat the B-skip picture as a B-picture, and performspecial play-back.

Note that the picture layer may indicate whether or not a picture is aB-skip picture. It is also possible to generate a B-skip picture as apicture which has the same pixel data of a reference picture immediatelyprior to the B-skip picture in display order. In such a case, the RAUmap MI may indicate whether the B-skip picture is predicted from areference picture immediately prior to the B-skip picture or a referencepicture immediately subsequent to the B-skip picture, in display order.

Third Embodiment

Furthermore, by recording a program for realizing the picture codingapparatus and the picture decoding apparatus described in the aboveembodiments, on a recording medium, such as a flexible disk, it ispossible to easily implement the processing described in the aboveembodiments by an independent computer system.

FIGS. 19A, 19B, and 19C are explanatory diagrams showing a case when thepicture coding apparatus and the picture decoding apparatus described inabove embodiments are realized by a computer system, by using a programwhich is recorded on a recording medium, such as a flexible disk.

FIG. 19B shows a front outside view of the flexible disk, across-sectional structure of the flexible disk, and a body of theflexible disk body. FIG. 19A shows an example of a physical format ofthe flexible disk body which is a main body of the recording medium. Theflexible disk body FD is equipped in a case F, and a plurality of tracksTr are formed on a surface of the disk from a circumference towards aninternal circumference in a shape of concentric circles, and each trackis segmented into sixteen sectors Se in an angle direction. Therefore,regarding the flexible disk storing the above program, the above programis recorded in an area allocated on the flexible disk body FD.

Furthermore, FIG. 19C shows a structure by which the flexible disk bodyFD records and play-backs the program. When the above program forrealizing the picture coding apparatus and the picture decodingapparatus is recorded on the flexible disk body FD, the program iswritten from a computer system via a flexible disk drive FDD.Furthermore, when processing performed by the picture coding apparatusand the picture decoding apparatus is structured in the program systemin the flexible disk, the program is read out from the flexible disk viathe flexible disk drive, and transferred to the computer system Cs.

Note that the above explanation has described to use the flexible diskas a recording medium, but it is possible to use an optical diskinstead. Note also that the recording medium is not limited to theabove, but may be anything for enabling to record the program, such asan IC card and a ROM cassette.

As described above, according to the present invention, the picturecoding apparatus adds the RAU map into a header of the RAU, and thepicture decoding apparatus refers to the added RAU map before decodingeach picture, so that it is possible to reduce decoding operations, andespecially to improve play-back quality of a packaged media, in which aspecial play-back function is crucial. Therefore, the present inventionhas a high practical value.

As described above, the present invention has been described by usingthe above embodiments and the respective variations, but the presentinvention is not limited to the above.

For example, each functional block shown in the block diagrams (FIGS. 10and 16, for example) is realized as a large scale integration (LSI)which is a typical integrated circuit. These functional blocks can beintegrated separately, or a part or all of them may be integrated into asingle chip (For example, functional blocks except a memory can beintegrated as a single chip.)

The integrated circuit can be called an IC, a system LSI, a super LSI oran ultra LSI depending on their degrees of integration.

The integrated circuit technique is not limited to the LSI, and it maybe implemented as a dedicated circuit or a general-purpose processor. Itis also possible to use a Field Programmable Gate Array (FPGA) that canbe programmed after manufacturing the LSI, or a reconfigurable processorin which connection and setting of circuit cells inside the LSI can bereconfigured.

Furthermore, if due to the progress of semiconductor technologies ortheir derivations, new technologies for integrated circuits appear to bereplaced with the LSIs, it is, of course, possible to use suchtechnologies to implement the enclosed functional blocks as anintegrated circuit. For example, biotechnology, organic chemicaltechnology, and the like can be applied to the above implementation.

Note that, among the functional blocks, only units for storing data tobe coded or decoded is not integrated into the chip, but realized as adifferent function.

INDUSTRIAL APPLICABILITY

The picture coding apparatus and picture decoding apparatus according tothe present invention can be applied, in play-backing the VC-1 streamand the like, to all devices which have a special play-back function,such as high-speed play-back, and is especially useful for optical diskperipheral devices in which the special play-back function is critical.

We claim:
 1. A picture coding apparatus which codes pictures,comprising: a picture coder configured to code each picture according toa picture type of the picture; a first coder configured to code firstsupplementary information included in a random accesser including codedpictures and indicating respective picture types of the coded picturesin a decoding order of the coded pictures; a second coder configured tocode second supplementary information included in the random accesserand indicating respective pieces of picture structure information of thecoded pictures in the decoding order; a third coder configured to codestuffing bits in the random accesser; a writer configured to write, at aposition prior to a starting picture in the random accesser, the firstsupplementary information coded by said first coder, the secondsupplementary information coded by said second coder, and the stuffingbits coded by said third coder; an audio coder configured to code audiodata; and a multiplexer configured to multiplex coded audio data andcoded information written in the random accesser, and to generate atransport stream, wherein the respective picture types of the codedpictures include at least an I picture on which intra-prediction codingis performed, a P picture on which inter-prediction coding is performedwith reference to one picture, a B picture on which inter-predictioncoding is performed with reference to two pictures, and a Skippedpicture which is to be displayed with an image of a reference picturepositioned immediately prior to a target picture in the coded picturesin the decoding order, the picture structure information of each of thecoded pictures includes information indicating whether each of the codedpictures is to be displayed as a three-field image or a two-field imageat a 3:2 pulldown, and Skipped pictures are sequentially arranged in thedecoding order to form a still picture sequence.
 2. A picture codingmethod of coding pictures, comprising: coding each picture according toa picture type of the picture; coding first supplementary informationincluded in a random accesser including coded pictures and indicatingrespective picture types of the coded pictures in a decoding order ofthe coded pictures; coding second supplementary information included inthe random accesser and indicating respective pieces of picturestructure information of the coded pictures in the decoding order;coding stuffing bits in the random accesser; writing, at a positionprior to a starting picture in the random accesser, the firstsupplementary information coded in said coding of the firstsupplementary information, the second supplementary information coded insaid coding of the second supplementary information, and the stuffingbits coded in said coding of the stuffing bits; coding audio data; andmultiplexing coded audio data and coded information written in therandom accesser, and generating a transport stream, wherein therespective picture types of the coded pictures include at least an Ipicture on which intra-prediction coding is performed, a P picture onwhich inter-prediction coding is performed with reference to onepicture, a B picture on which inter-prediction coding is performed withreference to two pictures, and a Skipped picture which is to bedisplayed with an image of a reference picture that is positionedimmediately prior to a target picture in the coded pictures in thedecoding order, the picture structure information of each of the codedpictures includes information indicating whether each of the codedpictures is to be displayed as a three-field image or a two-field imageat a 3:2 pulldown, and Skipped pictures are sequentially arranged in thedecoding order to form a still picture sequence.
 3. A picture decodingapparatus, comprising: a receiver configured to receive a transportstream in which coded audio data and coded information written in arandom accesser are multiplexed, the random accesser including codedpictures; a demultiplexer configured to demultiplex the transportstream, and to generate the coded audio data and the coded informationwritten in the random accesser; a detector configured to detect firstsupplementary information, second supplementary information, andstuffing bits which are stored at a position prior to a starting picturein the random accesser; a selector configured to select a picture to bedecoded, from the coded pictures in the random accesser, based on thefirst supplementary information and the second supplementaryinformation; a picture decoder configured to decode the picture selectedby said selector; and an audio decoder configured to decode the codedaudio data, wherein the first supplementary information is included inthe random accesser and indicates respective picture types of the codedpictures in a decoding order, the respective picture types of the codedpictures include at least an I picture on which intra-prediction codingis performed, a P picture on which inter-prediction coding is performedwith reference to one picture, c) a B picture on which inter-predictioncoding is performed with reference to two pictures, and a Skippedpicture which is to be displayed with an image of a reference picturethat is positioned immediately prior to a target picture in the codedpictures in the decoding order, the second supplementary information isincluded in the random accesser and indicates respective pieces ofpicture structure information of the coded pictures in the decodingorder, the picture structure information of each of the coded picturesincludes information indicating whether each of the coded pictures is tobe displayed as a three-field image or a two-field image at a 3:2pulldown, and Skipped pictures are sequentially arranged in the decodingorder to form a still picture sequence.
 4. A picture decoding method,comprising: receiving a transport stream in which coded audio data andcoded information written in a random accesser are multiplexed, therandom accesser including coded pictures; demultiplexing the trans ortstream and generating the coded audio data and the coded informationwritten in the random accesser; detecting first supplementaryinformation, second supplementary information, and stuffing bits whichare stored at a position prior to a starting picture in the randomaccesser; selecting a picture to be decoded, from the coded pictures inthe random accesser, based on the first supplementary information andthe second supplementary information; decoding the picture selected insaid selecting; and decoding the coded audio data, wherein the firstsupplementary information is included in the random accesser andindicates respective picture types of the coded pictures in a decodingorder, the respective picture types of the coded pictures include atleast an I picture on which intra-prediction coding is performed, a Ppicture on which inter-prediction coding is performed with reference toone picture, a B picture on which inter-prediction coding is performedwith reference to two pictures, and a Skipped picture which is to bedisplayed with an image of a reference picture that is positionedimmediately prior to a target picture in the coded pictures in thedecoding order, the second supplementary information is included in therandom accesser and indicates respective pieces of picture structureinformation of the coded pictures in the decoding order, the picturestructure information of each of the coded pictures includes informationindicating whether each of the coded pictures is to be displayed as athree-field image or a two-field image at a 3:2 pulldown, and Skippedpictures are sequentially arranged in the decoding order to form a stillpicture sequence.
 5. A method of recording a coded stream onto arecording medium, comprising: coding each picture according to a picturetype of the picture; coding first supplementary information included ina random accesser including coded pictures and indicating respectivepicture types of the coded pictures in a decoding order of the codedpictures; coding second supplementary information included in the randomaccesser and indicating respective pieces of picture structureinformation of the coded pictures in the decoding order; coding stuffingbits in the random accesser; writing the first supplementary informationcoded in said coding of the first supplementary information, the secondsupplementary information coded in said coding of the secondsupplementary information, and the stuffing bits coded in said coding ofthe stuffing bits, at a position prior to a starting picture in therandom accesser; coding audio data; multiplexing coded audio data andcoded information written in the random accesser, and generating atransport stream; and recording the transport stream onto the recordingmedium, wherein the respective picture types of the coded picturesinclude at least an I picture on which intra-prediction coding isperformed, a P picture on which inter-prediction coding is performedwith reference to one picture, a B picture on which inter-predictioncoding is performed with reference to two pictures, and a Skippedpicture which is to be displayed with an image of a reference picturethat is positioned immediately prior to a target picture in the codedpictures in the decoding order, the picture structure information ofeach of the coded pictures includes information indicating whether eachof the coded pictures is to be displayed as a three-field image or atwo-field image at a 3:2 pulldown, and Skipped pictures are sequentiallyarranged in the decoding order to form a still picture sequence.